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1.0  SUMMARY 


This  report  describes  improvements  in  the  Department  of  Defense  Human  Factors 
Analysis  and  Classification  System  (DoD  HFACS).  Military  services  had  a  unanimous  desire  to 
improve  DoD  HFACS  inter-rater  reliability.  A  working  group,  composed  of  representatives 
from  all  services,  met  several  times  with  the  goal  of  improving  inter-rater  reliability  while 
retaining  the  value  of  the  tool.  Additional  requirements  included  preserving  compatibility  with 
existing  databases  and  having  inter-operability  across  the  services.  The  steps  involved  included 
determining  which  “nanocodes”  were  rarely  or  never  used  and  collapsing  nanocodes  and 
rewriting  definitions  to  arrive  at  a  total  of  109  nanocodes,  reduced  from  147  nanocodes.  A  table, 
included  in  this  report,  allowing  for  the  ready  translation  of  old  codes  into  new  codes  pennits 
continued  analysis  of  data  already  collected.  The  authors,  in  collaboration  with  the  DoD  HFACS 
Working  Group,  then  developed  a  stepwise  checklist  to  systematically  guide  investigators 
through  consideration  of  nanocodes.  Researchers  tested  several  iterations  of  the  technique  using 
students  in  Air  Force  mishap  investigation  courses  to  gauge  inter-rater  reliability.  Student 
investigators  were  also  invited  to  offer  constructive  criticism  to  hone  checklist  questions.  While 
inter-rater  reliability  results  are  encouraging,  the  DoD  HFACS  Working  Group  has  additional 
work  to  accomplish  to  realize  the  goal  of  an  optimally  reliable  human  factors  taxonomy.  This 
report  contains  the  complete  DoD  HFACS  version  7.0  taxonomy  as  well  as  the  checklist  that  was 
generated  and  tested  with  groups  of  mishap  investigation  students.  Suggestions  for  future  efforts 
are  offered,  to  include  an  on-going  research  program. 

2.0  INTRODUCTION 

As  will  be  demonstrated,  “human  factors”  are  causal  in  a  majority  of  military  mishaps. 
This  report  also  reports  on  a  Department  of  Defense  (DoD)  effort  to  improve  the  system  used  to 
categorize  causal  and  contributing  human  factors.  Specifically,  recent  attempts  to  improve 
coding  methods  with  the  goal  of  achieving  better  inter-rater  reliability  and  ultimately  more 
actionable  recommendations  to  improve  safety  will  be  described. 

Hartmann,  in  a  widely  read  and  highly  regarded  article,  asserted  that  reliability  is  a 
necessary  but  not  a  sufficient  basis  for  validity.  Hartmann  went  on  to  specify  that  there  are  two 
methods  that  can  be  employed  to  determine  reliability:  percentage  agreement  reliability  and 
reliability  coefficient.  Hartmann  advocated  for  the  latter  over  the  former,  as  percentage 
agreement  may  produce  inflated  estimates  of  reliability  [1].  Another  issue  to  bear  in  mind  when 
considering  reliability  is  that  categories  must  be  mutually  exclusive  and  exhaustive  (that  is, 
contain  no  overlapping  elements  and  be  complete)  to  achieve  the  highest  reliability.  Overlapping 
elements  may  result  in  observers  using  different  categories  for  the  same  observation  and  thus 
finding  fewer  distinctions  between  entities  being  compared. 

The  roots  of  the  Human  Factors  Analysis  and  Classification  System  (HFACS)  are 
described  in  “A  Human  Error  Approach  to  Accident  Investigation:  The  Taxonomy  of  Unsafe 
Operations”  [2]  and  catalogued  in  a  Federal  Aviation  Administration  technical  report  [3]  and  a 
book,  A  Human  Error  Approach  to  Aviation  Accident  Analysis  [4].  Their  system  is  built  upon 
the  “Swiss  cheese”  model  of  Reason  [5],  Reason  recommended  that  a  mishap  investigation  start 
with  the  unsafe  acts,  which  represent  active  failure.  The  investigation  does  not  stop  there, 
however,  as  latent  failures  and  conditions  are  examined  next.  Latent  conditions  may  exist 
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undetected  and  unexpressed  for  years  and  include  preconditions,  unsafe  supervision,  and 
organizational  influences. 

According  to  Reason,  unsafe  acts  include  both  errors  and  violations  [5].  Errors  may  be 
skill-based  or  may  be  due  to  decisional  or  perceptual  factors.  Violations  may  be  routine  (such  as 
cutting  the  same  comers  that  many  others  cut)  or  exceptional.  Preconditions  for  unsafe  acts 
include  environmental  (physical  or  technological)  factors,  conditions  of  operators  (adverse 
mental  states  or  adverse  physiological  states  or  physical/mental  limitations),  or  personnel  factors 
(crew  resource  management  or  personal  readiness).  Reason  argued  that  it  is  also  essential  to 
investigate  at  the  supervisory  and/or  organizational  level  because  such  factors  have  direct  impact 
on  preconditions  [5].  Addressing  preconditions  is  likely  to  reveal  opportunities  to  improve 
safety. 

Unsafe  supervision  includes  inadequate  supervision,  supervisors  planning  inappropriate 
operations,  a  supervisor  failing  to  correct  a  known  problem,  and  supervisory  violations.  Finally, 
there  are  organization  influences,  to  include  resource  management,  organizational  climate,  and 
organizational  process.  One  way  to  conceptualize  these  categories  is  to  consider  them  “bins” 
containing  the  smaller  units. 

Table  1  depicts  the  results  of  a  query  of  the  Air  Force  Safety  Automated  System 
(AFSAS)  database,  which  is  accessed  via  a  secure  website,  for  fiscal  years  (FY)  2010  through 
2013  (1  October  2009  through  30  September  2013)  to  assess  overall  human  factors  involvement 
in  aviation  mishaps.  Table  2  shows  the  results  of  the  overall  involvement  of  human  factors  in 
ground  mishaps.  These  numbers  empirically  demonstrate  that  human  factors  are,  in  fact,  a  major 
concern  for  aviation  and  ground  safety. 

Table  1.  Aviation  Mishaps  for  FY  2010  -  2013 


Class3 

Total 

No. 

Aviation 

Mishaps 

Aviation 
Mishaps  with 
at  Least  1 
Human 
Factors  Code 

Percentage  of 
Aviation  Mishaps 
with  at  Least  1 
Human  Factors 
Code 

Total  No. 
Human 
Factors  Codes 

A 

129 

113 

87.60 

1,452 

B 

218 

113 

51.83 

754 

C 

2,518 

895 

35.54 

2,586 

D 

3,142 

737 

23.46 

1,179 

E 

28,803 

1,094 

3.8 

3,185 

Grand  Total 

34,810 

2,952 

59.59 

9,156 

aAs  defined  in  Air  Force  Instruction  (AFI)  91-204  [6].  See  Glossary. 


It  should  be  noted  that  DoD  HFACS  is  not  required  to  be  used  for  Classes  C,  D,  and  E 
mishaps  (highlighted).  The  reader  is  thus  cautioned  not  to  be  misled  by  the  lower  percentages 
and  the  deflating  impact  on  the  grand  total.  The  involvement  of  human  factors  is  therefore  likely 
heavily  underestimated  in  U.S.  Air  Force  (USAF)  mishaps,  particularly  Classes  C,  D,  and  E 
mishaps,  as  a  result. 
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Table  2.  Ground  Mishaps  for  FY  2010  -  2013 


Class3 

Total 

No. 

Ground 

Mishaps 

Ground 

Mishaps  with  at 
Least  1  Human 
Factors  Code 

Percentage  of 
Ground  Mishaps 
with  at  Least  1 
Human  Factors 
Code 

Total  No. 

Human 
Factors  Codes 

A 

216 

207 

95.83 

1,038 

B 

100 

81 

81.00 

311 

C 

13,985 

5,901 

42.20 

10,385 

D 

19,685 

4,069 

20.67 

5,556 

E 

859 

94 

10.94 

292 

Grand  Total 

34,845 

10,352 

38.73 

17,582 

aAs  defined  in  AFI  91-204  [6].  See  Glossary. 


Beaubien  and  Baker,  while  generally  favorable  in  their  review  of  HFACS,  note  that 
HFACS  is  a  bit  coarse  as  it  does  not  delineate  reasons  for  the  conditions  it  identifies.  Beaubien 
and  Baker  also  note  that  latent  failures  are  difficult  to  identify  in  mishap  analysis  [7].  The 
context  of  their  review  must  be  appreciated,  as  they  were  examining  coding  schemes  that  were 
used  with  data  already  collected.  Their  final  point  is  important:  HFACS  categories  are  nominal 
and  not  sequential  and  thus  do  not  reveal  a  chain  of  events.  Therefore,  they  do  not  differentiate 
causes  from  effects.  That  issue,  however,  is  relatively  easy  to  remedy  in  the  overall  scheme  of 
an  investigation.  For  example,  the  USAF  constructs  a  mishap  sequence  of  contributory  and 
causal  findings  and  embeds  DoD  HFACS  within  it.  O’Connor  noted  the  above  criticisms  and 
detailed  the  efforts  to  address  them,  to  include  the  formation  of  a  DoD  Working  Group  in  2003, 
which  created  DoD  HFACS  [8].  DoD  HFACS  introduced  increased  granularity,  an  additional 
level  of  classification:  “nanocodes.”  The  original  DoD  HFACS  included  147  nanocodes, 
organized  under  the  categories  (bins)  delineated  above  ( unsafe  acts,  preconditions,  unsafe 
supervision,  organization  influences).  O’Connor  examined  the  reliability  of  DoD  HFACS, 
version  6.2.  He  found  that  U.S.  Navy  and  Marine  Corps  aviators  undergoing  mishap  investigator 
training  were  unable  to  achieve  acceptable  reliability,  but  noted  that  they  had  received  only 
minimal  training  [8].  Although  the  raters  were  able  to  agree  on  the  nanocodes  not  used,  they 
were  unable  to  achieve  consistent  agreement  concerning  which  nanocodes  applied  (“there  were 
only  seven  nanocodes  in  which  50%  or  greater  of  the  participants  agreed  to  select  the  nanocode,” 
p.  602  [8]).  O’Connor  noted  that  raters  were  confused  by  the  number  (147)  of  available 
nanocodes  and  that  the  nanocodes  contained  overlapping  concepts.  He  found  that  collapsing 
codes  improved  inter-rater  reliability  and  therefore  argued  for  nanocodes  that  are  exhaustive, 
parsimonious,  and  mutually  exclusive.  O’Connor  also  noted  that  his  research  participants  may 
not  have  been  reading  and  considering  the  nanocodes’  one-paragraph  definitions,  relying  instead 
on  the  names  of  the  nanocodes  [8]. 

O’Connor  called  for  subject  matter  experts  to  review  the  nanocodes  to  determine  if  some 
could  be  removed  or  combined  with  other  nanocodes.  He  went  as  far  as  to  suggest  that  the 
nanocode  level  be  abandoned  if  acceptable  reliability  could  not  be  achieved  without  extensive 
training  [8].  A  201 1  Aerospace  Medical  Association  presentation,  “DoD  HFACS  X:  Inter-Rater 
Reliability,”  prepared  by  human  factors  practitioners  (Brian  T.  Musselman,  Jeffrey  D.  Alton, 
Thomas  G.  Hughes,  Patricia  LeDuc,  Richard  J.  Farley,  and  Antonio  B.  Carvalhais)  from  the 
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three  service  safety  centers,  had  four  expert  raters  code  54  USAF  Class  A  mishaps  with  DoD 
HFACS  6.2  [9].  They  found  a  kappa  coefficient  of  .5494  with  76  out  of  147  (52%)  nanocodes 
having  reliability  greater  than  or  equal  to  .60.  The  authors  recommended  “improved  code 
definition”  and  development  of  an  “organized  training  curriculum”  [9].  Subsequent  studies  used 
DoD  HFACS  X,  which  contained  fewer  nanocodes  (102,  rather  than  147).  The  average  kappa 
coefficient  increased  to  an  impressive  0.84  with  expert  coders,  but  novice  coders  continued  to 
struggle,  achieving  kappa  coefficients  of  .2453  and  .3239.  The  authors  urged  the  development  of 
a  decision  tree  algorithm,  redesign  of  DoD  HFACS  into  larger  buckets  (even  if  granularity  would 
be  sacrificed),  and  limiting  coding  at  the  nanocode  level  to  experts  only. 

The  steps  in  the  current  effort  to  improve  DoD  HFACS  included  detennining  the 
frequency  that  each  of  the  nanocodes  was  used  and  considering  retiring  those  nanocodes  that 
were  very  infrequently  used.  Nanocodes  that  were  similar  in  the  phenomena  they  described,  as 
evidenced  by  having  overlapping  definitions,  were  merged  and  the  definitions  reworked.  The 
goal  was  to  reduce  the  number  of  nanocodes  and  to  improve  the  mutual  exclusivity  of  the 
remaining  nanocodes.  Specifically,  AFSAS  was  further  queried  for  FY  2010  through  2013  for 
aviation  and  ground  mishaps  to  detennine  the  frequency  of  use  of  each  of  version  6.2’s  147 
nanocodes.  The  frequency  of  each  HFACS  nanocode  for  USAF  aviation  and  ground  mishaps  is 
depicted  in  Appendix  A.  It  should  be  noted  that  AFSAS  was  queried  to  arrive  at  two  totals.  The 
first  count  tallied  a  specific  HFACS  nanocode  cited  once  per  mishap  (labeled  “unique”  in  the 
tables).  Otherwise,  a  given  nanocode  assigned  against  multiple  members  of  a  crew  would  inflate 
the  total.  The  other  tally  counted  the  grand  total  of  HFACS  nanocodes  used,  with  no  restriction 
on  how  many  times  a  nanocode  was  used  in  any  given  mishap.  The  U.S.  Anny  and  Navy,  as 
members  of  the  DoD  HFACS  Working  Group,  performed  similar  tallies.  In  the  USAF,  for 
example,  PC  201  was  used  only  once  for  all  classes  of  aviation  and  ground  mishaps.  Finally,  the 
DoD  HFACS  Working  Group  ensured  that  nanocodes  were  aligned  in  the  correct  bins. 

Nanocodes  that  were  relocated  to  other  bins  were  reassigned  an  alphanumeric  to  be  consistent 
with  the  new  bin.  Ultimately,  the  147  nanocodes  in  version  6.2  were  collapsed  tol09  nanocodes 
in  version  7.0.  Table  3  provides  the  list  of  collapsed  (version  7.0)  nanocodes.  The  working 
group  then  developed  a  checklist,  colloquially  known  as  “Turbo  HFACS,”  that  uses  a  decision 
tree  to  guide  investigators  (see  Appendix  B).  A  response  of  “yes”  guides  the  investigator  to  the 
correct  “bin”  and  suggests  a  list  of  defined  nanocodes. 

This  report  delineates  the  motivation  to  change  DoD  HFACS  6.2  and  to  document  the 
changes  made  in  DoD  HFACS  7.0.  This  report  also  examines  the  inter-rater  reliability  of  DoD 
HFACS. 
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Table  3.  Conversion  Chart  for  DoD  HFACS  6.2  Nanocodes  Realigned  into  New  Bins  and 

Collapsed  in  DoD  HFACS  7.0 


Collapsed  DoD  HFACS  6.2 
Nanocodes 

Realigned  DoD  HFACS  6.2 
Nanocodes 

New  DoD  HFACS  7.0 
Nanocodes 

AE106 

AE103 

AE204 

AE203 

AE107 

AE301 

PC504 

OC002 

OCOOl 

OR002 

OR001 

OR004 

OP007 

OR006 

OSOOl 

OR007 

OS002 

PC201 

PC202 

PC210 

PC209 

PC211 

PC207 

PC212 

PC209 

PC213 

PC209 

PC214 

AE206 

PC301 

PC304 

PC303 

PC305 

PC306 

PC305 

PC308 

PC307 

PC309 

PC305 

PC313 

PC312 

PC403 

PC317 

PC316 

PP201 

PC318 

PP204 

PC319 

PC401 

PC405 

PC  109 

PC402 

PC405 

PC  109 

PC506 

PCI  10 

PC403 

PC317 

PC404 

PP201 

PC318 

PC506 

PCI  10 

PC509 

PC508 

PC510 

PC508 

PE  102 

PE101 

PE  104 

PE101 

PE  105 

PE101 

PE  107 

PE  106 

PE111 

PE101 

PP102 

PP108 

PP110 

PP109 

PP111 

AE201 

PP112 

PP108 

PP202 

PC302 

PP203 

PC302 

PP204 

PC319 

PP205 

PC307 

PP206 

PC305 

SF002 

SF001 

SI007 

SP004 

SP003 

SP005 

SI008 
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3.0 


METHODS 


3.1  Participants 

Three  hundred  forty  students  attending  USAF  aircraft  mishap  investigation  courses 
served  as  participants.  Most  of  the  participants  were  pilots  and  maintenance  personnel  attending 
the  Aircraft  Mishap  Investigation  Course  (AMIC)  at  Headquarters,  Air  Force  Safety  Center, 
Kirtland  Air  Force  Base,  NM.  Additional  data  were  collected  from  aerospace  medical  personnel 
(flight  surgeons,  aerospace  physiologists,  and  clinical  psychologists)  who  attended  the  Aircraft 
Mishap  Investigation  and  Prevention  course,  held  at  the  USAF  School  of  Aerospace  Medicine, 
Wright-Patterson  Air  Force  Base,  OH. 

3.2  Procedures 

Participants  were  given  approximately  45  minutes  to  read  and  code  sanitized  (basic 
identifying  information  had  been  removed)  synopses  of  mishap  reports  that  had  been 
investigated  by  Safety  Investigation  Boards  (SIBs).  The  synopses  were  approximately  two 
typed,  single-spaced  pages  in  length,  using  a  10-point  font.  To  protect  privilege,  all  mishap 
reports  were  immediately  collected  at  the  conclusion  of  the  exercises.  These  mishap  synopses 
are  not  published  here  because  the  degree  of  additional  sanitizing  that  would  have  been 
necessary  to  publish  them  in  this  report  would  have  rendered  them  virtually  incomprehensible. 

The  research  design  for  this  project  was  not  strictly  pre-planned,  but  rather  it  evolved  and 
capitalized  on  opportunities  that  presented  themselves.  Above  all,  it  was  a  proof  of  concept. 
Table  4  summarizes  the  evolution  of  the  research  activities.  A  more  detailed  account  follows. 

Table  4.  Summary  of  the  Evolution  of  DoD  HFACS  7.0  Research  Activities 


First  Trials  Second  Trials  Third  Trials 


Student  investigator  teams 
provided  Checklist  only 

Student  investigator  teams 
directed  to  use  only  DoD 
HFACS  7.0  for  exercise 

18-question  version  of 
checklist  used 


Student  investigator  teams 
given  answer  sheets  along 
with  checklist  and  required  to 
submit  responses  on  it 
Student  investigator  teams 
directed  to  use  DoD  HFACS 

6.2  and  then  introduced  to 
version  7.0  for  exercise 
8-question  (with  subquestions) 
version  of  checklist  used 


Student  investigator  teams 
given  answer  sheets  along 
with  checklist  and  required  to 
submit  responses  on  it 
Student  investigator  teams 
taught  to  use  DoD  HFACS  6.2 
and  then  introduced  to  version 
7.0  for  exercise 
8-question  (with  subquestions) 
version  of  checklist  used 


Student  investigator  teams 
asked  to  list  the  three  to  five 
(and  then  the  five)  most 
important  HFACS  nanocodes 


Student  investigator  teams 
asked  to  list  the  five  most 
important  HFACS  nanocodes 
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3.2.1  First  Trials.  In  the  first  data  collections,  31  student  investigator  teams  used  an  18-question 
version  of  the  DoD  HFACS  7.0  checklist  to  code  three  aircraft  mishap  scenarios.  The  lead 
researcher  (first  author)  presented  a  brief  introduction  (approximately  10  minutes)  to  the  DoD 
HFACS  7.0  checklist.  The  participants  were  directed  to  use  the  questions  of  the  checklist  and 
work  together  in  teams  of  two  or  three  members.  Following  the  advice  of  O’Connor  and  Walker 
[10],  participants  were  organized  into  small  teams  rather  than  working  alone  to  better  simulate 
the  conditions  of  an  SIB.  Participants  were  instructed  to  not  speak  to  any  member  of  another 
team  about  the  mishap  during  the  exercise. 

3.2.2  Second  Trials.  The  next  series  of  data  collection  aimed  to  directly  compare  DoD  HFACS 

6.2  to  DoD  HFACS  7.0.  Student  investigator  teams  were  given  a  mishap  scenario  and  directed 
to  first  use  version  6.2  as  outlined  in  AFI  91-204  [6].  Table  5  presents  the  outcome  as 
determined  by  the  actual  SIB  and  reviewed  by  the  Memorandum  of  Final  Evaluation  (MOFE). 
After  the  student  rater  teams’  responses  were  collected,  the  teams  were  trained  to  use  version  7.0 
(using  basically  the  same  introduction  described  above)  and  directed  to  again  code  the  scenario, 
without  regard  to  what  they  coded  using  version  6.2.  To  encourage  student  investigator  teams  to 
read  nanocode  definitions,  they  were  required  to  record  their  answers  on  sheets  that  only 
contained  the  alphanumeric  codes,  so  that  they  would  not  base  their  decisions  merely  on  the 
names  of  the  nanocodes  without  reading  and  considering  the  full  definition.  Moreover,  rater 
groups  were  asked  to  list  the  three  to  five  nanocodes  that  were  the  most  important  in  the  mishap, 
of  course  starting  with  those  that  they  deemed  causal.  Following  the  input  from  the 
epidemiologists  identified  in  the  Acknowledgments,  the  participants  were  ultimately  directed  to 
list  the  five  most  important  DoD  HFACS  nanocodes.  In  any  case,  the  reader  will  note,  as 
depicted  in  Table  5,  that  the  actual  SIB  and  the  respective  MOFE  found  12  DoD  HFACS 
nanocodes  to  be  applicable. 

Table  5.  Results  Determined  by  Actual  SIB  and  MOFE  for  Mishap  1 


Mishap-Level  Nanocodes  Person-Level  Nanocodes 

OP004  (Contributory)  AE105  (Causal) 

SI003  (Contributory)  PC  102  (Contributory) 

SP003  (Contributory)  PC307  (Contributory) 

SP006  (Causal)  PC308  (Contributory) 

PC504  (Causal) 

PC508  (Causal) 

PE  102  (Contributory) 
PP109  (Causal) 


3.2.3  Third  Trials.  Another  data  collection  was  held  using  a  mishap  that  had  been  coded  with 
fewer  nanocodes  by  the  SIB  and  that  included  only  nanocodes  that  transitioned  to  version  7.0. 
Because  the  purpose  of  AMIC  is  to  train  investigators  and  not  serve  as  a  research  laboratory,  this 
AMIC  class  received  more  detailed  instruction  on  a  strategy  to  use  DoD  HFACS  6.2.  The 
student  investigators  needed  to  be  prepared  to  investigate  mishaps  immediately  upon  the 
completion  of  their  training  and  there  was  no  start  date  yet  established  for  the  operational 
transition  to  version  7.0.  In  applying  version  6.2,  student  investigators  were  urged  to  read  and 
consider  definitions  rather  than  just  rely  on  the  one -page  wire  diagram.  The  following  person- 
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level  nanocodes  resulted  from  the  efforts  of  the  actual  SIB  and  MOFE  (no  mishap-level 
nanocodes  were  detennined): 

•  AE202  (causal) 

•  AE201  (causal) 

•  PC206  (causal) 

•  PC  102  (causal) 

•  AE105  (contributory) 

After  these  student  investigator  teams  completed  their  coding  with  version  6.2,  their  answer 
sheets  were  collected.  These  student  investigator  teams  were  then  introduced  to  version  7.0, 
using  basically  the  same  instruction  used  in  the  first  two  trials. 

3.2.4  Qualitative  Feedback.  The  feedback  received  from  students  led  the  authors  and  the  rest  of 
the  working  group  to  continually  refine  questions,  eventually  arriving  at  a  solution  of  eight 
questions  with  subquestions,  located  in  Appendix  C.  Students  were  subsequently  asked  to 
provide  written  feedback  on  their  opinions  of  the  changes  made  in  HFACS  7.0. 

4.0  RESULTS 

4.1  First  Trials 

During  the  first  series  of  data  collection,  18  of  31  (58%)  rater  teams  selected  the  identical 
“yes”  pattern  when  coding  Scenario  One  using  DoD  HFACS  7.0.  Four  of  the  3 1  (13%)  rater 
teams  selected  an  identical  but  alternate  pattern.  Twenty-four  (77%)  rater  teams  selected  the 
same  nanocodes  (AE105,  AE202,  and  PC  102)  as  the  top  three  (out  of  109)  overall  codes.  Fleiss’ 
kappa  in  considering  the  responses  to  the  18  questions  was  .847.  Fleiss’  kappa  for  the  109 
nanocodes  was  .545  and  the  average  pairwise  Cohen’s  kappa  was  .543. 

The  18-queston  version  of  DoD  HFACS  7.0  did  not  fare  as  well  with  two  other  scenarios. 
In  25  rater  teams  coding  Scenario  Two,  only  four  rater  teams  selected  an  identical  “yes”  pattern. 
There  were  two  other  common  patterns,  with  each  selected  by  two  rater  teams.  Twenty  (80%) 
rater  teams  selected  the  same  top  three  codes,  AE103,  SV004,  and  SI001.  Fleiss’  kappa  in 
considering  the  responses  to  the  18  questions  was  .498.  Fleiss’  kappa  for  the  109  nanocodes  was 
.415  and  the  average  pairwise  Cohen’s  kappa  was  .400. 

Scenario  Three  had  2  of  15  rater  teams  selecting  an  identical  “yes”  pattern.  Five  codes 
were  selected  10  or  more  times  by  rater  teams.  Fifteen  rater  teams  selected  the  top  three  overall 
codes:  AV003,  PC202,  and  PC204.  Fleiss’  kappa  in  considering  the  responses  to  the  18 
questions  was  .550.  Fleiss’  kappa  for  the  109  nanocodes  was  .487  and  the  average  pairwise 
Cohen’s  kappa  was  .512. 

4.2  Second  Trials 

As  seen  in  Table  6,  during  the  exercise  using  Mishap  1,  when  the  student  rater  teams  used 
DoD  HFACS  6.2,  9  out  of  14  rater  teams  (64%)  matched  at  least  one  of  the  above  findings  as 
being  among  their  most  important  three  to  five  DoD  HFACS  nanocodes.  One  rater  team  of  14 
(7%)  matched  three  nanocodes;  8  rater  teams  (57%)  had  one  match,  and  5  rater  teams  (36%)  had 
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no  matches  of  their  top  three  to  five  DoD  HFACS  nanocodes  to  those  of  the  SIB.  Using  DoD 
HFACS  7.0,  two  student  rater  teams  (14%)  had  three  matches;  three  student  rater  teams  (21%) 
had  two  matches,  nine  student  rater  teams  (64%)  had  one  match,  and  zero  student  rater  teams  had 
no  matches.  Making  this  contrast  even  more  stark  (and  more  favorable  to  version  7.0)  is  the  fact 
that  two  of  the  nanocodes  identified  in  Mishap  1  using  DoD  HFACS  6.2,  PC308  and  PE  102,  did 
not  transition  to  version  7.0  and  thus  were  not  available  to  the  raters  during  the  version  7.0 
portion  of  the  exercise. 

Table  6.  Comparing  DoD  HFACS  6.2  to  7.0  Anchored  Against 
Actual  SIB  Results,  Mishap  1 


No.  of  Matches 
to  Actual  SIB 

DoD  HFACS  6.2 

DoD  HFACS  7.0 

3 

1  student  rater  team  matched  SIB 

2  student  rater  teams  matched  SIB 

2 

3  student  rater  teams  matched  SIB 

1 

8  student  rater  teams  matched  SIB 

9  student  rater  teams  matched  SIB 

0 

5  student  rater  teams 

4.3  Third  Trials 

Table  7  presents  the  results  of  the  exercise  using  Mishap  2.  Two  rater  teams  elected  to 
list  only  four  codes  as  the  “most  significant”  during  the  version  7.0  portion  of  the  exercise  and 
could  not  be  persuaded  to  list  more.  By  doing  so,  they  lessened  the  opportunity  to  maximize 
matching  what  the  SIB  found. 

Table  7.  Comparing  DoD  HFACS  6.2  to  7.0  Anchored  Against 
Actual  SIB  Results,  Mishap  2 


No.  of  Student  Rater 
Team  Matches  to 
Actual  SIB 

DoD  HFACS  6.2 

DoD  HFACS  7.0 

4 

3  student  rater  teams  matched  SIB 

1  student  rater  team  matched  SIB 

3 

7  student  rater  teams  matched  SIB 

5  student  rater  teams  matched  SIB 

2 

4  student  rater  teams  matched  SIB 

5  student  rater  teams  matched  SIB 

1 

2  student  rater  teams  matched  SIB 

0 

1  student  rater  team  matched  SIB 

Finally,  student  raters  were  asked  to  provide  written  feedback  on  their  perception  of  the 
relative  value  of  version  7.0  over  version  6.2.  The  tables  in  Appendix  C  depict,  respectively, 
student  rater  team  comments  at  the  conclusion  of  the  exercises  using  Mishaps  1  and  2  as  well  as 
a  rejoinder,  which,  unfortunately,  was  not  offered  to  the  students  at  the  time  of  their  class 
attendance.  The  first  set  of  comments  is  mostly  neutral,  as  they  are  criticisms  of  both  versions. 
The  clearly  positive  comments,  however,  do  outnumber  the  clearly  negative  comments  by  five  to 
two.  The  second  set  of  comments  is  more  positive  (9  out  of  14,  with  no  negative  comments). 
Previously,  comments  from  students  were  collected  in  a  more  informal  fashion,  but  were  still 
useful  in  the  evolution  of  the  checklist.  Some  typical  themes  from  the  student  investigators 
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included  that  the  new  version  is  “less  intimidating”  and  “less  subjective”  and  gives  investigations 
structure.  Suggestions  for  improvement  included  the  observation  that  some  of  the  questions  are 
too  broad  and  the  subquestions  need  to  be  read  and  considered  even  if  the  instructions  advise 
users  to  skip  over  the  sub  questions. 

5.0  DISCUSSION 

In  a  series  of  comparisons  using  a  variety  of  mishap  scenarios,  the  checklist  for  DoD 
HFACS  7.0  perfonned  well.  These  encouraging  results  can  be  explained  as  follows:  A 
taxonomy  that  has  fewer  nanocodes  and  nanocodes  that  have  distinct  meaning  improves  user 
satisfaction  and  may,  itself,  increase  inter-rater  reliability.  While  a  systematic  approach  to 
considering  the  larger  categories  as  well  as  the  nanocodes  likely  is  a  key  component  to  the 
improvement  of  inter-rater  reliability,  simply  encouraging  student  investigators  to  consult  the 
definitions  of  the  nanocodes  also  likely  improved  inter-rater  reliability.  Systematically  guiding 
investigators  to  consider  all  nanocodes  will  increase  the  likelihood  that  the  definitions  of  the 
nanocodes  will  be  read  and  considered.  As  pointed  out  by  previous  researchers  as  noted  in  this 
report,  requiring  coding  at  a  finer  degree  of  granularity  requires  training  and  providing 
investigators  with  the  proper  resources,  such  as  a  checklist. 

As  noted  in  the  comments  of  our  participants,  another  issue  is  the  correct  structure  of  the 
checklist  questions.  Too  many  questions  are  likely  to  try  the  patience  of  investigators,  while 
fewer  questions  with  subquestions  run  the  risk  of  investigators  missing  significant  areas  that 
could  benefit  from  further  inquiry.  The  feedback  gleaned  from  the  students  who  graciously 
participated  in  this  research  suggests  that  investigators  would  be  wise  to  not  skim  over 
subquestions  after  answering  “no”  to  the  major  question.  While  the  DoD  HFACS  Working 
Group  should  consider  honing  the  questions  and  subquestions,  DoD  HFACS  7.0  is  a  step  in  the 
right  direction  according  to  student  feedback  and  the  results  obtained  in  this  study.  A  future 
revision  should  revise  the  questions  and  elevate  some  of  the  subquestions  to  free  standing 
questions.  Above  all,  any  strategy  that  gets  investigators  to  read  and  consider  the  definitions  of 
the  nanocodes  will  result  in  a  better  investigative  outcome.  The  “yes/no”  fonnat  of  the  questions 
in  version  7.0  results  in  a  clear  binning  (getting  in  the  ballpark  of  applicable  nanocodes).  Such 
binning  of  causes  and  contributing  factors  represents  an  advancement  in  investigations  with 
actionable  results,  as  it  allows  leaders  to  more  accurately  allocate  resources  to  reduce  future 
mishaps.  Even  if  there  is  some  disagreement  as  to  which  exact  nanocode  within  a  bin  is  the 
cause,  at  least  the  correct  bin  is  identified  and  proper  attention  is  paid  to  mitigation  of  a  major 
cause  or  contributing  factor  of  mishaps. 

Future  efforts  should  include  a  continued  refinement  of  the  questions,  as  noted  above,  as 
well  as  the  creation  of  a  small  set  of  questions  to  assist  Aviation  Safety  Action  Program  (ASAP) 
reporters  submit  reports  that  more  clearly  highlight  human  factors  issues.  ASAP  reports  are 
considered  “safety  without  the  mishap”;  thus,  better  use  of  DoD  HFACS  could  actually  help 
improve  safety.  Above  all,  investigators  in  training  in  all  services  must  be  given  ample 
opportunity  to  practice  investigating  and  coding  mishaps  during  the  “organized  training 
curriculum”  advocated  by  Mussehnan  et  al.  [9],  using  the  guided  approach  provided  by  the 
checklist  contained  in  Appendix  B. 

Coda:  The  Joint  Service  Safety  Council  approved  DoD  HFACS  7.0  in  May  2014.  The 
results  of  the  inter-rater  reliability  studies  were  shared  with  the  council,  along  with  the 
unanimous  recommendation  from  the  human  factors  practitioners  from  all  three  safety  centers 
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that  it  be  formally  adopted  and  operationally  fielded.  The  Joint  Service  Safety  Council  seemed 
to  be  particularly  encouraged  that  no  previously  collected  and  archived  data  will  be  lost  as  we 
transition  to  DoD  HFACS  7.0.  As  noted,  the  conversion  table  (Table  3)  included  in  this  report 
will  allow  backwards  translation  of  previously  coded  infonnation. 
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APPENDIX  A 


Frequency  of  HFACS  Nanocodes  for  Aviation  and  Ground  Mishaps 


Nanocode 

Class  A 
Aviation 
Causal 

Class  A 
Unique 
Aviation 
Causal 

Class  A 
Aviation 
Contributory 

Class  A 
Unique 
Aviation 
Contributory 

Class  A 
Ground 
Causal 

Class  A 
Unique 
Ground 
Causal 

Class  A 
Ground 
Contributory 

Class  A 
Unique 
Ground 
Contributory 

All  Classes, 
Aviation  & 
Ground 

All  Classes, 
Unique 
Aviation  & 
Ground 

AE101 

4 

4 

3 

3 

3 

3 

2 

2 

133 

129 

AE102 

29 

19 

10 

7 

- 

- 

- 

- 

302 

205 

AE103 

39 

29 

15 

12 

11 

11 

8 

7 

597 

536 

AE104 

9 

9 

9 

9 

22 

22 

19 

18 

505 

486 

AE105 

16 

9 

22 

13 

4 

4 

4 

4 

228 

189 

AE106 

1 

1 

- 

- 

- 

- 

- 

- 

33 

32 

AE201 

24 

12 

27 

18 

51 

41 

69 

53 

2319 

2135 

AE202 

11 
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***FOR  REFERENCE  USE  ONLY*** 


INTRODUCTION 


Human  error  remains  the  leading  cause  of  Air  Force  mishaps.  Mishaps  are  rarely  attributable  to  a  single  cause, 
but  are  often  the  end  result  of  a  series  of  errors.  This  can  be  depicted  visually  through  the  "Swiss  Cheese 
Model"  (see  below).  Root  cause  analysis  can  be  performed  in  many  different  ways,  but  it  always  comes  down 
to  first  asking  why  something  occurred.  Start  with  the  problem,  asking  what  prompted  the  problem  to  happen 
in  the  first  place.  Then  keep  taking  it  further  and  further  until  you  can  pinpoint  specific  processes,  policies,  or 
procedures  that  didn’t  work.  It  all  comes  down  to  asking  "why?"  until  you  see  a  pattern  in  the  problem.  This 
HFACS  flip  book  was  developed  as  a  tool  to  guide  in  root  cause  analysis,  and  can  also  be  used  to  develop 
Interview  questions,  determine  potential  risk-management  hazards,  and  detect  human  error  trends.  The 
concept  behind  the  Human  Factors  Analysis  and  Classification  (HFACS)  flip  book  is  to  provide  a  quick  reference 
guide  for  mishap  investigators.  This  booklet  demonstrates  a  hierarchical  approach  to  link  each  act  to  a 
precondition  and  to  a  supervisory  and/or  organizational  role. 

BENEFITS  OF  POD  HFACS 

1.  Structured  analysis  of  human  error 

-  Sophisticated,  complete... yet  operational 

-  Detects  error  patterns 

2.  Gets  to  the  "why"...  not  just  the  "what" 

-  More  insightful  root  cause  determination 

-  Better  investigator  analysis 

3.  A  new,  data-driven  approach 

-  Supports  research  across  the  Force 

-  Easily  applied  to  large  body  of  existing  data 

-  Easily  applied  to  new  mishaps  and  events 

4.  Can  be  used  for  more  than  Operational  purposes 

-  Can  be  as  a  risk-management  tool  for  brainstorming 

-  Can  help  develop  interview  questions 

-  Applies  to  both  on-duty  and  off-duty  mishaps 


"SWISS-CHEESE  MODEL’ 


l 
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ACTS  I 

Ql. 

Did  the  mlshaD  oerson(s)  make  a  oerformance-based  error? . 

....  □  Yes,  AE100,  go  to  p.4  &  02 
....  □  No.  go  to  Q2 

02. 

Was  a  mishao  Dersonfsl  actions  a  result  of  ooor  ludament  and/or  decision? . 

....  □  Yes,  AE200,  go  to  p.S  &  03 

....  □  No,  go  to  03 

03. 

Did  a  mishao  oersonlsl  violate  a  commonly  known  law  or  renulation? . 

□  Yes.  AV100.no tOD.6R.Q4 

. . . U  No,  go  to  Q4 

PRECONDITIONS 

Q4. 

Was  the  environment  a  factor  in  the  mishap? . . . — . U  Yes,  go  to  Q4a 


_ _ _ □  No,  go  to  Q5 

Q4a. 

Did  the  physical  environment  affect  the  mishap  person(s)? . .  L)  Yes,  PE  100,  go  to  p.7  &  Q4b 

... . . . LI  No.  go  to  Q4b 

Q4b. 

Did  the  technological  environment  affect  the  mishap  person(s)? . □  Yes,  PE  200,  go  to  p,8  &  Q5 

. □  No.  go  to  Q5 


05. 

Did  the  mishap  person(s)  mental,  sensory,  or  physical  state  contribute  to  the  mishap?....  □  Yes,  go  to  Q5a 

...  U  No.  go  to  Q6 

Q5a. 

Did  the  mishap  person  have  a  medical  or  physical  condition? . □  Yes,  PC300,  go  to  p.9  &  05 b 

. . . □  No,  go  to  Q5b 

QSb. 

Did  the  mishap  person(s)  state  of  mind  create  an  unsafe  situation? . □  Yes,  PC200,  go  to  p.10  &  Q5d 

. □  No,  go  to  Q5d 

Q5c. 

Was  sensory  information  misperceived  or  misunderstood?... . . . . □  Yes,  PC500,  go  to  p.ll  &  OSe 

. . . □  No.  go  to  Q5e 


QSd. 


Did  the  mishap  person(s)  mental  awareness  create  an  unsafe  situation? . □  Yes,  PC100,  go  to  p.  12  &  Q6 

_ _ ......  □  No,  go  to  Q6 

Q6. 

Did  challenges  with  teamwork  contribute  to  the  mishap? . . . . □  Yes,  PP100,  go  to  p.13  &  Q7 

. . . . . . . . □  No,  go  to  Q7 


SUPERVISION 


Q7. 

2 
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Did  supervision  or  supervisory  policies  contribute  to  the  mishap? . Li  Yes,  go  to  Q7a 

_ _ _ _ _ □  No.  go  to  Q8 

Q7a. 

Did  a  supervisor  violate  a  commonly  known  law  or  regulation? . U  Yes,  SV100,  go  to  p.14  &  Q7b 

. . . □  No.  go  to  Q7b 

Q7b. 

Did  supervision  fail  to  plan  or  assess  known  hazards? . □  Yes,  SP100,  go  to  p. IS  8  Q7c 

_ _ _ _ L)  No.  go  to  Q7e 

Q7c. 

Did  a  supervisor  or  supervision  prove  inadequate? . . . □  Yes,  SHOO,  go  to  p.16  8  Q8 

. . . . . □  No,  go  to  Q8 _ 


ORGANIZATIONAL  INFLUENCES 


Q8. 


Did  the  organization  or  organizational  policies  contribute  to  the  mishap? 


□ 

□ 


Yes,  go  to  Q8a 
No,  Finished! 


Q8a. 

Did  a  problem  with  resources  create  an  unsafe  situation? . 

Q8b. 

Was  selection  or  training  of  personnel  a  factor? . . . 


Q8c. 

Did  an  organizational  instruction  or  policy  create  an  unsafe  situation? 


□  Yes,  ORIOO,  go  to  p.17  8  Q8b 

□  No,  go  to  Q8b 

□  Yes,  OPIOO,  go  to  p.18  8  Q8c 
U  No.  go  to  Q8b 

□  Yes.  OllOO,  go  to  p.19  8  Q8d 

□  No,  go  to  Q8d 


QML 

Did  the  safety  cfimate/culture  contribute  to  an  unsafe  situation? . . . . . □  Yes,  OCIOO,  go  to  p.20 

. □  No,  Finishedl 
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•44FOR  REFERENCE  USE  ONLY*4* 

ACTS 

"Active  Failures  or  Actions" 

PERFORMANCE-BASED  ERRORS  (AE100):  are  factors  that  occur  when  a  specific  action  is  performed  in  a  manner  that 
leads  to  a  mishap. 


Unintended  Operation  of  Equipment 

AE101 

Checklist  Not  Followed  Correctly 

AE102 

Procedure  Not  Followed  Correctly 

AE103 

Over-Controlled/Under-Controlled  Aircraft/Vehicle 

AE 104 

Breakdown  in  Visual  Scan 

AE10S 

Rushed  or  Delayed  a  Necessary  Action 

AE  107 

AE101  Unintended  Operation  of  Equipment:  is  a  factor  when  an  individual's  movements  inadvertently  activate  or  deactivate 
equipment,  controls  or  switches  when  there  is  no  intent  to  operate  the  control  or  device.  This  action  may  be  noticed  or 
unnoticed  by  the  Individual. 

AE102  Checklist  Not  Followed  Correctly:  Is  a  factor  when  the  individual,  either  through  an  act  of  commission  or  omission, 
makes  a  checklist  error  or  fails  to  run  an  appropriate  checklist. 

AE103  Procedure  Not  Followed  Correctly:  is  a  factor  when  a  procedure  is  performed  incorrectly  or  accomplished  in  the  wrong 
sequence. 

AE104  Over-Controlled/Under-Controlled  Aircraft/Vehkle/System:  is  a  factor  when  an  individual  responds  inappropnately  to 
conditions  by  either  over-  or  under-controlling  the  aircraft/vehide/system.  The  error  may  be  a  result  of  preconditions  or  a 
temporary  failure  of  coordination. 

AE105  Breakdown  in  Visual  Scan:  is  a  factor  when  the  individual  fails  to  effectively  execute  visual  scan  patterns. 

AE107  Rushed  or  Delayed  a  Necessary  Action:  is  a  factor  when  an  individual  takes  the  necessary  action  as  dictated  by  the 
situation  but  performs  these  actions  too  quickly  or  too  slowly. 
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***FOR  REFERENCE  USE  ONLY*** 

JUDGMENT  &  DECISION-MAKING  ERRORS  (AE200):  are  factors  that  occur  when  an  individual  proceeds  as  intended,  yet 
the  plan  proves  inadequate  or  inappropriate  for  the  situation,  e.g.  "An  honest  mistake." 


Inadequate  Real-Time  Risk  Assessment 

AE201 

Failure  to  Prioritize  Tasks  Adequately 

AE202 

Ignored  a  Caution/Warning 

AE205 

Wrong  Choice  of  Action  During  an  Operation 

AE206 

AE201  Inadequate  Real-Time  Risk  Assessment:  is  a  factor  when  an  individual  fails  to  adequately  evaluate  the  risks  associated 
with  a  particular  course  of  action  and  this  faulty  evaluation  leads  to  inappropriate  decision-making  and  subsequent  unsafe 
situations. 

AE202  Failure  to  Prioritize  Tasks  Adequately:  is  a  factor  when  the  individual  does  not  organize,  based  on  accepted 
priontization  techniques,  the  tasks  needed  to  manage  the  immediate  situation. 

AE20S  Ignored  a  Caution/Warning:  is  a  factor  when  a  caution  or  warning  is  perceived  and  understood  by  the  individual  but  is 
ignored  by  the  individual. 

AE206  Wrong  Choice  of  Action  During  an  Operation  is  a  factor  when  the  Individual,  through  faulty  logic  or  erroneous 
expectations,  selects  the  wrong  course  of  action. 
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6 


VIOLATIONS  (AV100):  are  factors  when  the  Individual  intentionally  breaks  the  rules  and  instructions.  "Violations  are 
deliberate. ' 


Performs  Work-A/ound  Violation 

AVOOl 

Commits  Widespread/Routine  Violation 

AV002 

Extreme  Violation  -  Lack  of  Discipline 

AV003 

AVOOl  Performs  Work-Around  Violation:  is  a  factor  when  (he  consequences/risk  of  violating  published  procedures  was 
recognlred,  consciously  assessed  and  honestly  determined  by  the  Individual,  crew  at  team  to  be  the  best  course  of  action. 
Routine  "work-arounds"  and  unofficial  procedures  that  are  accepted  by  the  community  as  necessary  for  operations  are  also 
captured  under  this  code. 

AV002  Commits  Wldcsprcad/Routine  Violation:  is  a  factor  when  a  procedure  or  policy  violation  Is  systemic  in  a  unit/setting 
and  not  based  on  a  risk  assessment  for  a  specific  situation.  It  needlessly  commits  the  individual,  team,  or  crew  to  an  unsafe 
course-of-action.  These  violations  may  have  leadership  sanction  and  may  not  routinely  result  in  disciplinary/administrative 
action.  Habitual  violations  of  a  single  individual  or  small  group  of  individuals  within  a  unit  can  constitute  a  routine/widespread 
violation  if  the  violation  was  not  routinely  disciplined  or  was  condoned  by  supervisors. 

AV003  Extreme  Violation  -  Lack  of  Discipline:  is  a  factor  when  an  individual,  crew  or  team  intentionally  violates  procedures  or 
policies  without  cause  or  need.  These  violations  are  unusual  or  isolated  to  specific  individuals  rather  than  larger  groups.  There 
is  no  evidence  of  these  violations  being  condoned  by  leadership.  These  violations  may  also  be  referred  to  as  "exceptional 
violations." 
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***FOR  REFERENCE  USE  ONLY*** 

PRECONDITIONS 
“Latent  Failures  or  Conditions " 

ENVIRONMENT:  The  environment  surrounding  a  mishap  is  the  physical  or  technological  factors  that  affect  practices, 
conditions,  and  actions  of  individual(s). 

PHYSICAL  ENVIRONMENT  (PC100):  are  factors  such  as  weather,  dimate,  fog,  brownout  (dust  or  sand  storm)  or  whiteout 
(snow  storm)  that  affect  the  actions  of  individual. 


Environmental  Conditions  Affecting  Vision 

PE101 

Vibration  Affects  Vision  or  Balance 

PE  103 

Heat/Cold  Stress  Impairs  Performance 

PE  106 

External  Force  or  Object  Impeded  an  Individual's  Movement 

PE  108 

Lights  of  Other  Vehide/Vessel/Aircraft  Affected  Vision 

PE  109 

Noise  Interference 

PE  110 

PE101  Environmental  Conditions  Affecting  Vision:  is  a  factor  that  includes  obscured  windows;  weather,  fog,  hate,  darkness; 
smoke,  etc.;  brownout/whiteout  (dust,  snow,  water,  ash  or  other  particulates);  or  when  exposure  to  windblast  affects  the 
individual's  ability  to  perform  required  duties. 

PE103  Vibration  Affects  Vision  or  Balance:  is  a  factor  when  the  Intensity  or  duration  of  the  vibration  is  sufficient  to  cause 
impairment  of  vision  or  adversely  affect  balance. 

PE  106  Heat/Cold  Stress  Impairs  Performance;  is  a  factor  when  the  individual  is  exposed  to  conditions  resulting  in 
compromised  performance. 

PE108  External  Force  or  Object  Impeded  an  Individual's  Movement:  is  a  factor  when  acceleration  forces  greater  than  one 
second  cause  injury  or  prevent/interfere  with  the  performance  of  normal  duties.  Do  not  use  this  code  to  capture  G-induced 
loss  of  consciousness. 

PE109  Lights  of  Other  Vehicle/Vessel/ Aircraft  Affected  Vision:  is  a  factor  when  the  absence,  pattern,  intensity  or  location  of 
the  lighting  of  other  vehicle/vessel/aircraft  prevents  or  interferes  with  safe  task  accomplishment. 

PEI  10  Noise  Interference:  is  a  factor  when  any  sound  not  directly  related  to  information  needed  for  task  accomplishment 
interferes  with  the  Individual's  ability  to  perform  that  task. 
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***FOR  REFERENCE  USE  ONLY*** 

TECHNOLOGICAL  ENVIRONMENT  (PE200):  are  factors  when  automation  or  the  design  of  the  workspace  affects  the 
actions  of  an  individual. 


Seat  and  Restraint  System  Problems 

PE201 

Instrumentation  and  Warning  System  Issues 

PE  202 

Visibility  Restrictions  (not  weather  related) 

PE  203 

Controls  and  Switches  are  Inadequate 

PE204 

Automated  System  Creates  an  Unsafe  Situation 

PE20S 

Workspace  Incompatible  with  Operation 

PE206 

Personal  Equipment  Interference 

PE  207 

Communication  Equipment  Inadequate 

PE  208 

PE201  Seat  and  Restraint  System  Problems:  is  a  factor  when  the  design  of  the  seat  or  restraint  system,  the  ejection  system  or 
seat  comfort  has  poor  impact-protection  qualities, 

PE 202  Instrumentation  and  Warning  System  Issues:  is  a  factor  when  instrument  factors  such  as  design,  reliability,  lighting, 
location,  symbology,  site,  display  systems,  auditory  or  tactile  situational  awareness  or  warning  systems  create  an  unsafe 
situation. 

PE203  Visibility  Restrictions  (not  weather  related):  is  a  factor  when  the  lighting  system,  windshield/windscreen/canopy 
design,  or  other  obstructions  prevent  necessary  visibility.  This  includes  glare  or  reflections  on  the 

windshield/windscreen/canopy.  Visibility  restrictions  due  to  weather  or  environmental  conditions  are  captured  under  PE101. 

PE204  Controls  and  Switches  are  Inadequate:  is  a  factor  when  the  location,  shape,  size,  design,  reliability,  lighting  or  other 
aspect  of  a  control  or  switch  are  inadequate 

PE205  Automated  System  Creates  an  Unsafe  Situation:  is  a  factor  when  the  design,  function,  reliability,  symbology,  logic  or 
other  aspect  of  automated  systems  creates  an  unsafe  situation. 

PE206  Workspace  Incompatible  with  Operation:  is  a  factor  when  the  workspace  is  incompatible  with  the  task  requirements 
and  safety  for  an  individual 

PE207  Personal  Equipment  Interference:  is  a  factor  when  the  individual's  personal  equipment  interferes  with  normal  duties  or 
safety. 

PE208  Communication  Equipment  Inadequate:  is  a  factor  when  communication  equipment  is  inadequate  or  unavailable  to 
support  task  demands.  This  includes  electronically  or  physically  blocked  transmissions.  Communications  can  be  voice,  data  or 
multi-sensory. 
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•**FOR  REFERENCE  USE  ONLY*** 

PHYSICAL  AND  MENTAL  STA  TE:  The  mentol  and  physical  stotes  of  individuals  are  how  people  know,  think,  learn, 
understand,  perceive,  feel,  hurt,  guess,  recognize,  notice,  wont,  wish,  hope,  decide,  expect,  remember,  forget,  imagine, 
ond  believe. 

PHYSICAL  PROBLEM  (PC300):  are  medical  or  physiological  conditions  that  can  result  in  unsafe  situations. 


Substance  Effects  (alcohol,  supplements,  medications,  drugs) 

PC  302 

Loss  of  Consciousness  (sudden  or  prolonged  onset) 

PC304 

Physical  Illness/Injury 

PC305 

Fatigue 

PC  307 

Trapped  Gas  Disorders 

PC310 

Evolved  Gas  Disorders 

PC311 

Hypoxla/Hyperventllation 

PC312 

Inadequate  Adaptation  to  Darkness 

PC314 

Dehydration 

PC31S 

Body  Site/Movement  Limitations 

PC317 

Physical  Strength  &  Coordination  (inappropriate  for  task  demands) 

PC318 

Nutntion/Diet 

PC319 

PC302  Substance  Effects  (alcohol,  supplements,  medications,  drugs):  is  a  factor  when  the  individual  uses  legal  or  illegal  drugs, 
supplements,  energy  drinks  or  any  other  substance  with  measurable  effect  that  interferes  with  performance 

PC304  Loss  of  Consciousness  (sudden  or  prolonged  onset):  is  a  factor  when  the  individual  has  a  loss  of  functional 
ca  pacity/consciousness  due  to  G-LOC,  seizure,  trauma  or  any  other  cause. 

PC305  Physical  Illness/Injury:  Is  a  factor  when  a  physical  Illness,  in|ury,  deficit  or  diminished  physical  capability  causes  an  unsafe 
situation.  This  includes  preexisting  and  operationally  related  medical  conditions,  over  exertion,  motion  sickness,  etc. 

PC307  Fatigue:  is  a  factor  causing  diminished  physical/mental  capability  resulting  from  chronic  or  acute  periods  of  prolonged 
wakefulness,  sleep  deprivation,  jet  lag.  shift  work  or  poor  sleep  habits 

PC310  Trapped  Gas  Disorders:  is  a  factor  when  gasses  in  the  middle  ear,  sinuses,  teeth  or  intestinal  tract  expand  or  contracts. 

PC311  Evolved  Gas  Disorders:  is  a  factor  when  Inert-gas  evolves  in  the  blood  causing  an  unsafe  situation.  This  Includes  chokes,  CNS, 
bends,  paresthesias  or  other  conditions  caused  by  inert -gas  evolution 

PC312  Hypoxia/Hyperventilation:  is  a  factor  when  the  individual  has  insufficient  oxygen  supply  to  the  body  and/or  breathing  above 
physiological  demands  causes  impaired  function. 

PC314  Inadequate  Adaptation  to  Darkness:  is  a  factor  when  the  normal  human  limitation  of  dark-adaptation  rate  affects  safety,  for 
example,  when  transitioning  between  aided  and  unaided  night  vision. 

PC315  Dehydration:  is  a  factor  when  the  performance  of  the  individual  is  degraded  due  to  dehydration  as  a  result  of  excessive  fluid 
losses  due  to  heat  stress  or  due  to  Insufficient  fluid  Intake. 

PC317  Body  Sire/Movement  limitations:  is  a  factor  when  the  site,  strength,  dexterity,  mobility  or  other  biomechanical  limitations 
of  an  individual  creates  an  unsafe  situation.  It  must  be  expected  that  the  average  individual  qualified  for  that  duty  position  could 
accomplish  the  task  in  question. 

PC318  Physical  Strength  &  Coordination  (inappropriate  for  task  demands):  is  a  factor  when  the  relative  physical  strength  and/ 
or  coordination  of  the  Individual  is  not  adequate  to  support  task  demands. 

PC319  Nutrition/Diet:  rs  a  factor  when  the  individual's  nutritional  state  or  poor  dietary  practices  are  inadequate  to  fuel  the  brain 
and  body  functions  resulting  In  degraded  performance. 
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•**FOR  REFERENCE  USE  ONLY*** 

STATE  OF  MIND  (PC200):  are  factors  when  an  individual's  personality  traits,  psychosocial  problems,  psychological 
disorders  or  inappropriate  motivation  creates  an  unsafe  situation. 


Psychological  Problem 

PC  202 

Life  Stressors 

PC203 

Emotional  State 

PC204 

Person  ality  Style 

PC205 

Overconfidence 

PC206 

Pressing 

PC207 

Complacency 

PC208 

Motivation 

PC209 

Mentally  Exhausted  (Burnout) 

PC?  1  s 

PC202  Psychological  Problem:  is  a  factor  when  the  individual  met  medical  criteria  tor  a  psychiatric  disorder. 

PC203  Life  Stressors:  is  a  factor  when  the  individual's  performance  is  affected  by  life  circumstance  problems  (includes 
relationship  issues,  financial  stressors,  recent  move,  etc.). 

PC204  Emotional  State:  is  a  factor  when  the  individual  is  under  the  influence  of  a  strong  positive  or  negative  emotion  and  that 
emotion  interferes  with  duties. 

PC205  Personality  Style:  is  a  factor  when  the  individual's  personal  interaction  with  others  creates  an  unsafe  situation. 

Examples  are  authoritarian,  over-corvservatlve,  impulsive,  invulnerable,  submissive  or  other  personality  traits  that  result  In 
degraded  performance. 

PC206  Overconfidence:  Is  a  factor  when  the  individual  overvalues  or  overestimates  personal  capability,  the  capability  of  others 
or  the  capability  of  aircraft/vehicles  or  equipment. 

PC207  Pressing:  is  a  factor  when  the  individual  knowingly  commits  to  a  course  of  action  that  excessively  presses  the  individual 
and/or  their  equipment  beyond  reasonable  limits  (e.g.,  pushing  self  or  equipment  too  hard). 

PC 208  Complacency:  is  a  factor  when  the  individual  has  a  false  sense  of  security,  is  unaware  of,  or  ignores  hazards  and  is 
inattentive  to  risks. 

PC209  Motivation:  is  a  factor  when  the  individual's  motivation  to  accomplish  a  task/mission  is  excessive,  weak,  indecisive  or 
when  personal  goals  supersede  the  organization’s  goals. 

PC215  Mentally  Exhausted  (Burnout):  Is  a  factor  when  the  individual  has  the  type  of  exhaustion  associated  with  the  wearing 
effects  of  high  operational  and/or  lifestyle  tempo  in  which  operational  requirements  impinge  on  the  ability  to  satisfy  personal 
requirements  and  leads  to  degraded  effectiveness. 
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***FOR  REFERENCt  USE  ONLY*** 

SENSORY  MISPERCEPTION  (PC500):  are  factors  resulting  in  degraded  sensory  inputs  (visual,  auditory  or  vestibular)  that 
create  a  misperception  of  an  object,  threat  or  situation. 


Motion  Illusion  -  Kinesthetic 

PC501 

Turning/Balance  Illusion  -  Vestibular 

PC502 

Visual  Illusion 

PC  503 

Misperception  of  Changing  Environment 

PC  504 

Misinterpreted/Misread  Instrument 

PC505 

Misinterpretation  of  Auditory/Sound  Cues 

PC  50  7 

Spatial  Disorientation 

PC  508 

Temporal/Time  Distortion 

PC511 

PCS01  Motion  Illusion  -  Kinesthetic:  is  a  factor  when  physical  sensations  of  the  ligaments,  muscles  or  joints  cause  the 
individual  to  have  an  erroneous  perception  of  orientation,  motion  or  acceleration,  (if  this  illusion  leads  to  spatial  disorientation 
you  must  code  PC 508. 1 

PCS02  Tuming/Balance  Illusion  -  Vestibular:  is  a  factor  when  stimuli  acting  on  the  balance  organs  in  the  middle  ear  cause  the 
individual  to  have  an  erroneous  perception  of  orientation,  motion  or  acceleration.  (If  this  illusion  leads  to  spatial  disorientation 
you  must  code  PCS08.I 

PC503  Visual  Illusion:  is  a  factor  when  visual  stimuli  result  in  an  erroneous  perception  of  orientation,  motion  or  acceleration.  (If 
this  illusion  leads  to  spatial  disonentation  you  must  code  PC508.) 

PC504  Misperception  of  Changing  Environment:  is  a  factor  when  an  individual  rmsperceives  or  misjudges  altitude,  separation, 
speed,  closure  rate,  road/sea  conditions,  aircraft/vehicle  location  within  the  performance  envelope  or  other  operational 
conditions. 

PC505  Mlslnterpreted/Mlsread  Instrument:  is  a  factor  when  the  individual  Is  presented  with  a  correct  Instrument  reading  but 
its  significance  is  not  recognized,  It  is  misread  or  is  misinterpreted. 

PCS07  Misinterpretation  of  Auditory/Sound  Cues:  is  a  factor  when  the  auditory  inputs  are  correctly  interpreted  but  are 
misleading/disorienting  or  when  the  inputs  are  incorrectly  interpreted  and  cause  an  impairment  of  normal  performance. 

PC508  Spatial  Disorientation:  is  a  factor  when  an  individual  fails  to  correctly  sense  a  position,  motion,  or  attitude  of  the 
aircraft/vehicle/vessel  or  of  oneself.  Spatial  Disorientation  may  be  unrecognized  and/or  result  in  partial  or  total  incapacitation. 

PC511  Temporal/Time  Distortion:  is  a  factor  when  the  individual  experiences  a  compression  or  expansion  of  time  relative  to 
reality.  This  is  often  associated  with  a  "fight  or  flight"  response. 
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•**FOR  REFERENCE  USE  ONLY*** 

MENTAL  AWARENESS  (PC100):  are  factors  of  an  attention  management  or  awareness  failure  that  affects  the  perception 
or  performance  of  individuals. 


Not  Paying  Attention 

PC101 

Fixation 

PC  102 

Task  Over-Saturation/Under-Saturation 

PC103 

Confusion 

PC  104 

Negative  Habit  Transfer 

PC  105 

Distraction 

PC  106 

Geographically  Lost 

PC  107 

Interference/Interruption 

PC  108 

Technical  or  Procedural  Knowledge  Not  Retained  after  Training 

PC  109 

Inaccurate  Expectation 

PC110 

PC101  Not  Paying  Attention:  is  a  factor  when  there  is  a  lack  of  state  of  alertness  or  a  readiness  to  process  immediately 
available  information.  The  individual  has  a  state  of  reduced  conscious  attention  due  to  a  sense  of  security,  self-confidence, 
boredom  or  a  perceived  absence  of  threat  from  the  environment.  This  may  often  be  a  result  of  highly  repetitive  tasks. 

PC102  Fixation:  is  a  factor  when  the  individual  is  focusing  all  conscious  attention  on  a  limited  number  of  environmental  cues  to 
the  exclusion  of  others. 

PC103  Task  Over-Saturation/Under-Saturation:  is  a  factor  when  the  quantity  of  information  an  individual  must  process 
exceeds  their  mental  resources  in  the  amount  of  time  available  to  process  the  information. 

PC104  Confusion:  is  a  factor  when  the  Individual  is  unable  to  maintain  a  cohesive  and  orderly  awareness  of  events  and 
required  actions  and  experiences  a  state  charactented  by  bewilderment,  lack  of  clear  thinking  or  (sometimes)  perceptual 
disorientation, 

PC105  Negative  Habit  Transfer:  is  a  factor  when  the  individual  reverts  to  a  highly  learned  behavior  used  in  a  previous  system 
or  situation  and  that  response  is  inappropriate  for  current  task  demands. 

PC106  Distraction;  is  a  factor  when  the  individual  has  an  interruption  of  attention  and/or  inappropriate  redirection  of  attention 
by  an  environmental  cue  or  mental  process. 

PC107  Geographically  Lost:  is  a  factor  when  the  individual  is  at  a  different  location  from  where  one  believes  they  are. 

PC108  Interference/Interruption:  is  a  factor  when  an  Individual  is  performing  a  highly  automated/learned  task  and  is 
distracted  by  another  cue/event  that  results  in  the  Interruption  and  subsequent  failure  to  complete  the  original  task  or  results 
in  skipping  steps  in  the  original  task. 

PC109  Technical  or  Procedural  Knowledge  Not  Retained  after  Training:  Is  a  factor  when  the  individual  falls  to  absorb/retain 
required  information  or  is  unable  to  recall  past  experience  needed  for  safe  task  completion. 

PC110  Inaccurate  Expectation:  is  a  factor  when  the  individual  expects  to  perceive  a  certain  reality  and  those  expectations 
are  strong  enough  to  create  a  false  perception  of  the  expectation. 
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•••FOB  REFERENCE  USE  ONLY*** 

TEAMWORK  (PP100):  factors  refer  to  interactions  among  individuals,  crews,  and  teams  involved  with  the  preparation 
and  execution  of  a  task/mission  that  resulted  in  human  error  or  an  unsafe  situation. 


Failure  of  Crew/Team  leadership 

PP101 

Inadequate  Task  Delegation 

PP103 

Rank/Position  Intimidation 

PP104 

lack  of  Assertiveness 

PP105 

Critical  Information  Not  Communicated 

PP1D6 

Standard/Proper  Terminology  Not  Used 

PP107 

Failed  to  Effectively  Communicate 

PP108 

Task/Mission  Planning/Briefing  Inadequate 

PP109 

PP101  Failure  of  Crew/Team  leadership:  is  a  factor  when  the  crew/team  leadership  techniques  failed  to  facilitate  a  proper 
crew/team  climate,  to  include  establishing  and  maintaining  an  accurate  and  shared  understanding  of  the  evolving  task  and  plan 
on  the  part  of  all  crew/team  members. 

PP103  Inadequate  Task  Delegation:  is  a  factor  when  the  crew/team  members  failed  to  actively  manage  the  distribution  of 
tasks  to  prevent  the  overloading  of  any  individual  member. 

PP104  Rank/Position  Intimidation:  is  a  factor  when  the  differences  in  rank  of  the  team/crew  caused  the  task  performance 
capabilities  to  be  degraded.  Also,  conditions  where  formal  or  informal  authority  gradient  is  too  steep  or  too  flat  across  a 
crew/team  and  this  condition  degrades  collective  or  individual  performance. 

PP105  lack  of  Assertiveness:  is  a  factor  when  an  Individual  failed  to  state  critical  information  or  solutions  with  appropriate 
persistence  and/or  confidence. 

PP106  Critical  Information  Not  Communicated:  is  a  factor  when  known  cntical  Information  was  not  provided  to  appropriate 
individuals  in  an  accurate  or  timely  manner. 

PP107  Standard/Proper  Terminology  Not  Used:  is  a  factor  when  dear  and  concise  terms,  phrases,  hand  signals,  etc.  per 
service  standards  and  training  were  not  used. 

PP108  Failed  to  Effectively  Communicate:  is  a  factor  when  communication  is  not  understood  or  is  misinterpreted  as  the  result 
of  behavior  of  either  sender  or  receiver.  Communication  failed  to  include  backing  up,  supportive  feedback  or 
acknowledgement  to  ensure  that  personnel  correctly  understood  announcements  or  directives. 

PP109  Task/Mission  Planning/Briefing  Inadequate:  is  a  factor  when  an  individual,  crew  or  team  failed  to  complete  all 
preparatory  tasks  associated  with  planning/briefing  the  task/mission. 
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***FOR  REFERENCE  USE  ONLY*** 

SUPERVISION 

"Direct  Supervisory  Chain  of  Command “ 

SUPERVISION:  Supervision  is  a  factor  in  a  mishap  if  the  methods,  decisions  or  policies  of  the  supervisory  chain  of 
command  directly  affect  practices,  conditions  or  actions  of  the  individual(s). 

SUPERVISORY  VIOLATIONS  (SV100):  are  factors  when  supervisors  willfully  disregard  instructions  or  policies. 


Failure  to  Enforce  Existing  Rules  (supervisory  act  of  omission) 

SV001 

Allowing  Unwritten  Policies  to  Become  Standard 

SV002 

Directed  Individual  to  Violate  Existing  Regulations 

SV003 

Authorized  Unqualified  Individuals  for  Task 

SV004 

SV001  Failure  to  Enforce  Existing  Rules  (supervisory  act  of  omission):  is  a  factor  when  unit  (organizational)  and  operating  rules 
have  not  been  enforced  by  a  supervisor. 

SV002  Allowing  Unwritten  Policies  to  Become  Standard:  is  a  factor  when  unwritten  or  "unofficial"  policy  is  perceived  and 
followed  by  the  Individual,  although  it  has  not  been  formally  recognized  by  the  organization, 

SV003  Directed  Individual  to  Violate  Existing  Regulations:  is  a  factor  when  a  supervisor  directs  a  subordinate  to  violate 
existing  regulations,  instructions  or  technical  guidance. 

SV004  Authorized  Unqualified  Individuals  for  Task:  is  a  factor  when  an  individual  has  not  met  the  general  training 
requirements  for  the  job/weapon  system  and  is  considered  non-current  but  supervision/leadership  inappropriately  allows  the 
individual  to  perform  the  task  for  which  the  individual  is  non-current. 
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•**FOR  REFERENCE  USE  ONLY*** 

PLANNED  INAPPROPRIATE  OPERATIONS  (SP100):  are  factors  when  supervision  fails  to  adequately  plan  or  assess  the 
hazards  associated  with  an  operation  and  allows  for  unnecessary  risk. 


Directed  Task  Beyond  Personnel  Capabilities 

SPOOl 

Inappropriate  Team  Composition 

SP002 

Selected  Individual  with  Lack  of  Current  or  Limited  Experience 

SP003 

Performed  Inadequate  Risk  Assessment  -  Formal 

SPOOb 

Authorized  Unnecessary  Hazard 

SPOO/ 

SP001  Directed  Task  Beyond  Personnel  Capabilities:  is  a  factor  when  supervisor/management  directs  personnel  to  undertake 
a  task  beyond  their  skill  level  or  beyond  the  capabilities  of  their  equipment. 

SP002  Inappropriate  Team  Composition:  is  a  factor  when  the  makeup  of  the  crew/team  should  have  reasonably  raised  safety 
concerns  in  the  minds  of  members  involved  in  the  task,  or  in  any  other  individual  directly  related  to  the  scheduling  of  this  task. 

SP003  Selected  Individual  with  Lack  of  Current  or  Limited  Experience:  is  a  factor  when  the  supervisor  selects  an  individual 
whose  experience  Is  not  sufficiently  current  or  proficient  to  permit  safe  task  execution. 

SP006  Performed  Inadequate  Risk  Assessment  -  Formal:  is  a  factor  when  supervision  does  not  adequately  evaluate  the  risks 
associated  with  a  task  or  when  pre-mission  risk  assessment  tools/programs  are  inadequate. 

SP007  Authorized  Unnecessary  Hazard:  is  a  factor  when  supervision  authorizes  an  activity  or  task  that  is  unnecessarily 
hazardous  without  sufficient  cause  or  need. 
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•••FOB  REFERENCl  USE  ONLY*** 

INADEQUATE  SUPERVISION  (SHOO):  are  factors  when  department-level  or  command-level  supervision  proves 
inappropriate  or  improper  and/or  fails  to  identify  hazards;  recognize  and  control  risk;  provide  guidance,  training  and/or 
oversight. 


Supervisory/Command  Oversight  Inadequate 

SIOOl 

Improper  Role- Modeling 

SI002 

Failed  to  Provide  Proper  Training 

SI003 

Failed  to  Provide  Appropriate  Policy/Guidance 

51004 

Personality  Conflict  with  Supervisor 

SI005 

Lack  of  Supervisory  Responses  to  Critical  Information 

SI006 

failed  to  Identify/Correct  Risky  or  Unsafe  Practices 

SI007 

Selected  Individual  with  Lack  of  Proficiency 

SI008 

51001  Supervisory/Command  Oversight  Inadequate:  is  a  factor  when  the  availability,  competency,  quality  or  timeliness  of 
leadership,  supervision  or  oversight  does  not  meet  task  demands.  Inappropriate  supervisory  pressures  are  also  captured  under 
this  code. 

51002  Improper  Role-Modeling:  Is  a  factor  when  the  Individual’s  learning  Is  Influenced  by  the  behavior  of  supervisors  and  when 
that  learning  manifests  itself  in  actions  that  are  either  inappropriate  to  the  individual's  skill  level  or  violate  standard 
procedures. 

51003  Failed  to  Provide  Proper  Training:  is  a  factor  when  one  time  or  recurrent  training  programs,  upgrade  programs, 
transition  programs  or  any  other  local  training  is  inadequate  or  unavailable,  etc.  (Note:  the  failure  of  an  individual  to  absorb  the 
training  material  in  an  adequate  training  program  does  not  indicate  a  training  program  problem.) 

51004  Failed  to  Provide  Appropriate  Policy/Guidance:  is  a  factor  when  policy/guidance  or  tack  of  a  policy/guidance  leads  to  an 
unsafe  situation. 

51005  Personality  Conflict  with  Supervisor:  is  a  factor  when  a  supervisor  and  individual  member  experience  a  "personality 
conflict"  that  leads  to  a  dangerous  error  in  judgment/action. 

SM306  lack  of  Supervisory  Responses  to  Critical  Information:  is  a  factor  when  information  critical  to  a  potential  safety  issue 
was  provided  but  supervisory/management  personnel  failed  to  act  upon  it  (failure  to  close  the  loop) 

51007  Failed  to  Identify/Correct  Risky  or  Unsafe  Practices:  Is  a  factor  when  a  supervisor  fails  to  identify  or  correct  risky 
behaviors  or  unsafe  tendencies  and/or  fails  to  institute  remedial  actions.  This  includes  hazardous  practices,  condibons  or 
guidance. 

51008  Selected  Individual  with  lack  of  Proficiency:  is  a  factor  when  a  supervisor  selects  an  individual  that  is  not  proficient  in  a 
task,  mission  or  event. 
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***FOR  REFERENCE  USE  ONLY*** 

ORGANIZATIONAL  INFLUENCES 
" Upper-Level  Management,  Command  Level “ 

ORGANIZATION:  An  organization  is  the  communications,  actions,  omissions  or  policies  of  upper  level  management  that 
directly  or  Indirectly  affect  supervisory  practices,  conditions  or  actions  of  the  operotor(s). 

RESOURCE  PROBLEMS  (ORIOO):  are  factors  when  processes  or  policies  influence  system  safety,  resulting  in  inadequate 
error  management  or  creating  an  unsafe  situation. 


Command  and  Control  Resources  are  Deficient 

OROOl 

Inadequate  Infrastructure 

OROOB 

Failure  to  Remove  Inadequate/Worn-Out  Equipment  in  a  Timely  Manner 

OR005 

Failure  to  Provide  Adequate  Operational  Information  Resources 

OR008 

Failure  to  Provide  Adequate  Funding 

OR009 

OROOl  Command  and  Control  Resources  are  Deficient:  is  a  factor  when  installation  resources  are  inadequate  for  safe 
operations.  Examples  include:  command  and  control,  airfield  services,  battlegroup  management,  etc. 

OR003  Inadequate  Infrastructure:  Is  a  factor  when  support  facilities  (dining,  exercise,  quarters,  medical  care,  etc.)  or 
opportunity  for  recreation  or  rest  are  not  available  or  adequate.  This  includes  situations  where  leave  is  not  taken  for  reasons 
other  than  the  Individual's  choree. 

OROOS  Failure  to  Remove  Inadequatc/Worn-Out  Equipment  in  a  Timely  Manner:  Is  a  factor  when  the  process  through  which 
equipment  is  removed  from  service  is  inadequate. 

OR008  Failure  to  Provide  Adequate  Operational  Information  Resources:  is  a  factor  when  weather,  intelligence,  operational 
planning  material  or  other  information  necessary  for  safe  operations  planning  are  not  available. 

OR009  Failure  to  Provide  Adequate  Funding:  is  a  factor  when  an  organization  or  operation  does  not  receive  the  financial 
resources  to  complete  its  assigned  task/mission. 
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***FOR  REFERENCE  USE  ONLY*** 

PERSONNEL  SELECTION  &  TRAINING  (OSOOO):  are  factors  if  personnel  management  processes  or  policies,  directly  or 
indirectly,  influence  system  safety  and  results  in  poor  error  management  or  creates  an  unsafe  situation. 


Personnel  Recruiting  and  Selection  Policies  are  Inadequate 

OSOOl 

Failure  to  Provide  Adequate  Manning/Staffing  Resources 

OS002 

OSOOl  Personnel  Recruiting  and  Selection  Policies  are  Inadequate:  is  a  factor  when  the  process  through  which  Individuals  are 
screened,  brought  into  the  service  or  placed  Into  specialties  is  inadequate, 

OS002  Failure  to  Provide  Adequate  Manning/Staffing  Resources:  is  a  factor  when  the  process  through  which  manning, 
staffing  or  personnel  placement  or  manning  resource  allocations  are  inadequate  for  task/mission  demands. 
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•••FOR  REFERENCE  USE  ONLY*** 

POLICY  AND  PROCESS  ISSUES  (OPOOO):  are  factors  if  these  processes  negatively  influence  performance  and  result  in 
an  unsafe  situation. 


Pace  of  Ops-tempo/Workload 

OPOOl 

Organizational  Program/Policy  Risks  not  Adequately  Assessed 

OP002 

Provided  Inadequate  Procedural  Guidance  or  Publications 

OP003 

Organizational  (formal)  Training  is  Inadequate  or  Unavailable 

OP004 

Flawed  Doctrine/Philosophy 

OP005 

Inadequate  Program  Management 

OP006 

Purchasing  or  Providing  Poorly  Designed  or  Unsuitable  Equipment 

OP007 

OPOOl  Pace  of  Ops-tempo/Workload:  >s  a  factor  when  the  pace  of  deployments,  workload,  additional  duties,  off-duty 
education,  PMt  or  other  workload-inducing  conditions  of  an  individual  or  unit  creates  an  unsafe  situation. 

OPO02  Organliational  Program/Policy  Risks  not  Adequately  Assessed:  is  a  factor  when  the  potential  risks  of  a  large  program, 
operation,  acquisition  or  process  are  not  adequately  assessed. 

OPO03  Provided  Inadequate  Procedural  Guidance  or  Publications:  is  a  factor  when  written  direction,  checklists,  graphic 
depictions,  tables,  charts  or  other  published  guidance  is  inadequate,  misleading  or  inappropriate. 

OP004  Organizational  (formal)  Training  is  Inadequate  or  Unavailable:  is  a  factor  when  one-time  or  initial  training  programs, 
upgrade  programs,  transition  programs  or  other  training  that  is  conducted  outside  the  local  unit  is  inadequate  or  unavailable. 

OPOOS  Flawed  Doctrine/Philosophy:  is  a  factor  when  the  doctrine,  philosophy  or  concept  of  operations  in  an  organization  is 
flawed  or  accepts  unnecessary  risk  which  leads  to  an  unsafe  situation  or  unmitigated  hazard. 

OP006  Inadequate  Program  Management:  is  a  factor  when  programs  are  implemented  without  sufficient  support,  oversight  or 
planning. 

OP007  Purchasing  or  Providing  Poorly  Designed  or  Unsuitable  Equipment:  is  a  factor  when  the  processes  through  which 
aircraft,  vehicle,  equipment  or  logistical  support  are  acquired  allows  inadequacies  or  when  design  deficiencies  allow 
inadequacies  in  the  acquisition. 
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•**FOR  REFERENCE  USE  ONLY*** 

ClIMATE/CULTURE  INFLUENCES  (OCOOO):  are  factors  where  the  working  atmosphere  within  the  organization  influences 
individual  actions  resulting  in  human  error,  (e.g.  command  structure,  policies  and  working  environment). 


Organizational  Culture  (attitude/actions)  Allows  for  Unsafe  Task/Mission 

OCOOl 

Organizational  Over-confidence  or  Under-confidence  in  Equipment 

OC003 

Unit  Mission/ Aircraft/Vehide/Equipment  Change  or  Unit  Deactivation 

OC004 

Organizational  Structure  is  Unclear  or  Inadequate 

OC005 

OCOOl  Organizational  Culture  (attitude/actions)  Allows  for  Unsafe  Task/Mission:  a  factor  when  explkit/implicit  actions, 
statements  or  attitudes  of  unit  leadership  set  unit/organizational  values  (culture)  that  allow  an  environment  where  unsafe 
task/mission  demands  or  pressures  exist. 

OC003  Organizational  Over-confidence  or  Under-confidence  in  Equipment:  is  a  factor  when  there  is  organizational  over-  or 
under-confidence  in  an  aircraft,  vehicle,  device,  system  or  any  other  equipment. 

OC004  Unit  Mission/ Aircraft/Vehicle/Equipment  Change  or  Unit  Deactivation:  is  a  factor  when  the  process  of  changing 
missions,  aircraft/vehide/equipment  or  an  impending  unit  deactivation  creates  an  unsafe  situation. 

OCOOS  Organizational  Structure  is  Unclear  or  Inadequate:  is  a  factor  when  the  chain  of  command  of  an  Individual  or  structure 
of  an  organization  is  confusing,  non-standard  or  inadequate  and  this  creates  an  unsafe  situation. 
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APPENDIX  C 


Comments  Offered  by  Student  Raters  with  Rejoinders  at  Conclusion  of  Mishap  Exercises 

Mishap  1  Exercise 


Comments  Made  by  Student  Raters  after 
Coding  a  Mishap  Using  Both  Coding 
Schemes 

Rejoinder  (after  Reading  Written 
Comments) 

Overall 

Impression 

Much  easier  to  follow  the  process. 

Like  definition  at  bottom  of  page. 

We  made  definitions  more  accessible  to 
increase  the  probability  that  they  would  be 
read  and  considered. 

Positive 

1  think  the  definitions  are  too  subjective  to 
get  inter-rater  reliability.  In  order  to  prevent 
SIBs  from  coding  things  badly  you  need  to 
take  it  out  of  their  hands.  You  won’t  get  a 
reliable  system  with  subjective  definitions 
unless  you  limit  the  number  of  people 
coding  the  HF  data. 

It  is  a  trade-off.  While  a  cadre  of  consistent 
coders  would  increase  the  inter-rater 
reliability,  the  validity  would  not  necessarily 
increase,  as  we  are  hopeful  that  the  questions 
will  help  to  guide  the  investigation. 

Neutral 

1  liked  this  product  because  it  is  less 
subjective  and  much  more  logical  to  go 
through,  rather  than  just  picking  definition 
that  fit. 

Positive 

Good,  helps. .  .however  someone  unfamiliar 
with  all  the  codes  &  how  they  are 
categorized  under  the  major  groupings  has 
to  read  the  individual  definitions  to  make  a 
judgment  about  the  question  asked... So  in 
fact  you  are  adding  confusion/workload 
rather  than  decreasing  it. 

A  fair  point.  On  the  other  hand,  if  we  are 
forcing  pilots  to  consider  a  broader  range  of 
HFACS  nanocodes,  then  so  much  the  better. 

An  example  of  putting  the  compulsivity  of 
pilots  to  good  use. 

Slightly  Positive  to 
Slightly  Negative 

Q5  as  an  example,  I  would  break  up  the 
various  conditions  rather  than  ask  “could  it 
be  all  or  any  of  these.” 

A  fair  point.  We’ve  yet  to  achieve  the  right 
balance  of  questions  and  subquestions. 

Neutral 

Would  be  extremely  more  efficient  for  those 
with  more  experience  in  microcodes.  I  had 
to  review  all  the  codes  to  make  sure  1  didn’t 
miss  anything  (i.e.,  Q5). 

Reviewing  the  codes,  which  have  been  vastly 
reduced,  while  perhaps  inefficient,  is  not  the 
worst  possible  outcome  (see  above  reply). 

My  only  complaint  is  that  you  have  to  go 
read  the  codes  ahead  of  time  to  answer  the 
questions. 

Neutral 

As  an  investigating  officer,  1  did  not  feel 
confident  to  immediately  answer  “no”  per 
the  factors/headers  presented,  as  they  can  be 
subjective.  So  we  ended  up  answering 
“yes”  and  went  to  the  appropriate  pages 
“just  to  verify”  that  we  were  not 
overlooking  a  factor  that  was  involved. 

Neutral 

Q5  if  select  no,  then  many  questions  are 
skipped.  Mental,  physical,  or  sensory  is 
vague. 

Looks  like  the  students  are  really  concerned 
about  question  5.  We  will  look  into  that. 

Negative 
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Comments  Made  by  Student  Raters  after 
Coding  a  Mishap  Using  Both  Coding 
Schemes 

Rejoinder  (after  Reading  Written 
Comments) 

Overall 

Impression 

The  top  tier  questions  are  vague  to  the  point 
of  an  expected  reduction  in  HFAC  entries 
because  of  unawareness  of  included  lower 
tier  items. 

Negative 

Good  basic  idea. 

Maybe  make  questions  provide  more  clues 
to  HFACS  definition  in  its  category. 

The  issue  with  this  suggestion  is  the  risk  of 
making  questions  too  narrow.  For  example,  if 
one  concept  is  illustrated  in  the  definition,  the 
other  concepts  not  illustrated  are  at  risk  for 
being  overlooked. 

Positive 

This  was  much  easier  +  clearer  process 
w/the  guide  book  to  work  with.  This  is  a 
better  product  -  Impliment  (sic). 

Positive 

Recommendations- 

Layman  -  Some  of  the  questions  are  very 
similar  to  a  non-medical  person. 

Not  sure  what  this  means. 

Neutral 

Q8b  -  Codes  forget  selection  -  not  as  much 
on  training. 

See  next  comment  and  reply. 

Q8b  relates  mostly  or  solely  to  selection  and 
not  training. 

02  Differentiate  between  judgment  and 
decision.  Define  each. 

There  are  only  two  nanocodes  under  this 
subquestion: 

OR006  Personnel  Recruiting  and 
Selection  Policies  are  Inadequate 
OR007  Failure  to  Provide  Adequate 
Manning/Staffing  Resources 

The  students  have  a  point  and  we  will  address 
this  issue. 

Absolutely.  Those  words  are  underlined 
because  they  will  be  hyperlinked  with  their 
definitions  when  the  product  becomes 
electronic.  In  any  case,  currently  the 
nanocodes  and  their  definitions  provide 
differentiation. 

Neutral 

-  Easier  to  use. 

-  Need  to  place  the  definitions  of  the  under¬ 
lined  words  in  the  questions  under  the 
questions. 

See  comment  about  hypertext  and  underlined 
words  above. 

Positive 

Q4  &  Q5  (Overhead  Questions)  Can/could 
be  dropped  and  just  make  the  Q4a,  Q4b, 

Q5a,  Q5b,  etc.  overhead  questions  or  do  the 
above  suggestion  w/the  definitions. 

Yes,  go  to  AE100  p.  4  and  then  Q2. 

Suggest  to  read  yes,  go  to  (AE 

Series)  nanocodes  and  then  Q2. 

Neutral 
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Mishap  2  Exercise 


Comments  Made  by  Student  Raters  after  Coding 
a  Mishap  Using  Both  Coding  Schemes 

Rejoinder  (after  Reading  Written 
Comments) 

Overall 

Impression 

-  Recommend  rephrasing  Q5  to  also  focus  attention 
on  the  MPs  state  of  mind  going  into  the  mishap  sortie 

-  We  perceived  the  question  as  an  inquiry  into 
(among  other  things)  any  mental  problems  as 
opposed  to  state  of  mind  (which  isn’t  addressed  until 
a  subquestion). 

Yes,  we  clearly  need  to  reconsider  the 
subquestions. 

Neutral 

Consider  putting  examples  next  to  the  question. 

We  have  considered  using  examples.  Our 
hesitancy  to  do  so  is  that  the  examples 
cannot  be  inclusive  enough  and  thus  are 
likely  to  prematurely  steer  investigators  away 
from  a  full  consideration  of  the  question. 

Much  better  system,  still  very  subjective  in  that 
different  groups  will  have  different  answers. 

That  is  the  point  of  the  inter-rater  reliability 
study,  to  determine  the  extent  of  the 
subjectivity  problem. 

Positive 

Defining  the  key  words  underlined  would  help 
people  better  understand  the  question, 
i.e.,  Q4b:  actually  define  “technological 
environment.” 

Agreed,  underlined  words  will  be  hyper- 
linked  to  their  definitions. 

Neutral 

Better  for  research  processes  &  reference  points. 

Positive 

Def  of  each  term  helpful 

DoD  HFACS  7.0  checklist  is  awesome!! 

Positive 

pg  2  -  Q5b  -  “Yes,  PC200,  go  to  p.  10  &  Q5d”  - 
should  reference  Q5c. 

Positive 

pg  2  -  Q5c  -  “Yes,  PC500,  go  to  p.  1 1  &  Q5c”  - 
should  reference  Q5d. 

-Much  more  efficient,  simpler. 

-Crew  chief  proof. 

Easier  to  organize  with  this  method.  However,  the 
question  headings  do  not  give  one  a  good 
representation  of  the  subcategories.  We  found 
ourselves  reading  the  subcategories  to  see  if  the  main 
question  applied. 

We  are  finding  this  strategy  commonly 
employed. 

Positive 

The  booklet  is  a  fantastic  tool. 

-A  lot  easier  and  a  lot  less  intimidating. 

Positive 

This  mishap  seems  too  simple  to  “nail”;  there  can  be 
so  many  different  interpretations  on  how  to  solve  the 
problem. 

Neutral 

Our  team  felt  like  using  the  second  method  (7.0) 
allowed  us  to  work  small  to  big  and  gave  the 
assessment  some  structure.  Method  was  laid  out 
better  and  allowed  quicker  references  and  definitions. 

Positive 

Some  of  the  HFACS  are  redundant,  which  can  cause 
confusion  on  the  best  choice. 

Yes,  we  are  working  on  reducing,  if  not 
eliminating,  redundancy. 

Positive 

Good  to  have  definition  present. 
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Comments  Made  by  Student  Raters  after  Coding 
a  Mishap  Using  Both  Coding  Schemes 

Rejoinder  (after  Reading  Written 
Comments) 

Overall 

Impression 

Some  of  the  codes  are  still  redundant  (PC  106  & 

108). 

Agreed.  We  will  continue  to  refine  the 
codes. 

Neutral 

Some  of  the  main  questions  don’t  highlight  every 
aspect  of  the  subquestions:  i.e.,  Q4  only  made  me 
think  of  physical  environment.  If  I  marked  no  and 
skipped  to  Q5, 1  would  not  have  considered 
technological  environment. 

Yes,  the  subquestions  need  work. 

In  order  to  cover  all  possibilities,  you  have  to 
disregard  the  main  questions  and  look  at  all 
possibilities  in  that  category.  That  makes  the  main 
questions  not  only  irrelevant,  but  also  wasted 
effort/time. 

As  noted  above,  we  are  finding  this  strategy 
commonly  employed. 

The  format  surely  makes  the  choices  for  the  codes 
easier,  but  the  selection  process  is  subjective  in 
nature. 

Positive 

Every  subHFAC  could  be  Y  or  NO  questions  also 
and  the  codes  could  be  generated  easily  from  the 
answers  with  a  simple  software, 

Along  with  the  software  program,  the  database  could 
offer  a  100-question  survey. 

Seems  as  though  that  would  be  a  labor- 
intensive  process,  even  with  fewer  codes  to 
consider.  Not  sure  many  investigators  would 
comply. 

Neutral 
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LIST  OF  ABBREVIATIONS  AND  ACRONYMS 


AFI 

AMIC 

AFSAS 

ASAP 

DoD 

FY 

HFACS 

MOFE 

SIB 

USAF 


Air  Force  instruction 
Aircraft  Mishap  Investigation  Course 
Air  Force  Safety  Automated  System 
Aviation  Safety  Action  Program 
Department  of  Defense 
fiscal  year 

Human  Factors  Analysis  and  Classification  System 
Memorandum  of  Final  Evaluation 
Safety  Investigation  Board 
U.S.  Air  Force 
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GLOSSARY 


Causes  -  Deficiencies  that,  if  corrected,  would  likely  have  prevented  or  mitigated  damage  and/or 
injury.  Cause  does  not  imply  blame.  Events/conditions  that  are  highly  probable  results  of  other 
events/conditions  are  not  causes.  They  should  be  rated  as  either  contributors  or  outcomes. 

Class  A  Mishap  -  The  resulting  total  cost  of  damages  to  Government  and  other  property  is  $2 
million  or  more,  a  DoD  aircraft  is  destroyed  (excluding  UAS  [unmanned  aircraft  systems] 

Groups  1,  2,  or  3),  or  an  injury  or  occupational  illness  results  in  a  fatality  or  permanent  total 
disability. 

Class  B  Mishap  -  The  resulting  total  cost  of  damages  to  Government  and  other  property  is 
$500,000  or  more,  but  less  than  $2  million.  An  injury  or  occupational  illness  results  in  pennanent 
partial  disability,  or  when  three  or  more  personnel  are  hospitalized  for  inpatient  care  (which,  for 
mishap  reporting  purposes  only,  does  not  include  just  observation  or  diagnostic  care)  as  a  result 
of  a  single  mishap. 

Class  C  Mishap  -  The  resulting  total  cost  of  property  damages  to  Government  and  other 
property  is  $50,000  or  more,  but  less  than  $500,000,  or  a  nonfatal  injury  or  illness  results  in  1  or 
more  days  away  from  work,  not  including  the  day  of  the  injury. 

Class  D  Mishap  -  The  resulting  total  cost  of  property  damage  is  $20,000  or  more,  but  less  than 
$50,000,  or  a  recordable  injury  or  illness  not  otherwise  classified  as  a  Class  A,  B,  or  C  mishap.  A 
Class  D  mishap  is  any  nonfatal  injury  or  occupational  illness  that  does  not  meet  the  definition  of 
lost  time.  These  are  cases  where,  because  of  injury  or  occupational  illness,  the  employee  only 
works  partial  days,  has  restricted  duties  or  was  transferred  to  another  job,  and  required  medical 
treatment  greater  than  first  aid.  Loss  of  consciousness  (not  including  G-induced  loss  of 
consciousness,  which  is  considered  Class  E)  is  considered  a  Class  D  mishap  when  it  is  a  direct 
result  of  a  nonfatal  injury  or  occupational  illness. 

Class  E  Event  -  An  unplanned  occurrence,  or  series  of  occurrences,  that  does  not  meet  the 
reporting  criteria  of  a  mishap. 

Contributors  -  Single  events/conditions  that  are  essential  to  the  mishap  sequence.  They  offer  an 
independent  contribution  or  allow  the  progression  of  other  events/conditions.  If  an 
event/condition  is  both  contributory  and  causal,  rate  it  only  as  causal. 

Factor  -  Any  deviation,  out-of-the-ordinary,  or  deficient  action  or  condition  discovered  in  the 
course  of  a  mishap  investigation  that  in  the  board’s  opinion  contributed  to  the  eventual  outcome. 
Detennining  mishap  factors  (and  eliminating  non-factors)  enables  the  investigators  to  focus  the 
investigation  from  all  the  issues  under  examination  to  those  specific  areas  that  are  significant  in 
the  mishap  sequence.  Factors  explain  why  causes,  such  as  pilot  error,  supervision,  or  equipment 
failure,  occurred.  Factors  are  not  mutually  exclusive  but  are  often  interrelated  and  in  some  cases 
influence  each  other. 
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Non-Factors  Worthy  of  Discussion  (NFWODs)  -  NFWODs  typically  fall  into  one  of  three 
categories:  areas  uncovered  during  the  investigation  that  did  not  cause  the  mishap  or  influence 
the  outcome  but  should  be  fixed  due  to  the  potential  to  be  a  factor  in  a  future  mishap  (e.g., 
incorrect  information  in  a  maintenance  TO  [technical  order]),  areas  that  were  thoroughly 
investigated  and  subsequently  ruled  out  as  factors  (to  provide  context  to  the  audience  on  why 
these  areas  are  not  factors),  and  areas  that  may  be  considered  an  interest  item  to  the  convening 
authority  (e.g.,  risk  management,  crew  resource  management,  etc.).  NFWODs  are  the  source  for 
Other  Findings  and  Recommendations  of  Significance. 
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